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Abstract.  We  argue  that  one  of  the  early  goals  of  color  vision  is  to  distinguish  one  kind 
of  material  from  another.  Accordingly,  we  show  that  when  a  pair  of  image  regions  is  such 
that  one  region  has  greater  intensity  at  one  wavelength  than  at  another  wavelength,  and  the 
second  region  has  the  opposite  property,  then  the  two  regions  are  likely  to  have  arisen  from 
distinct  materials  in  the  scene.  We  call  this  material  change  circumstance  the  “opposite 
slope  sign  condition."  With  tins  criterion  as  a  foundation,  we  construct  a  reoresentation  of 
spectral  information  that  facilitates  the  recognition  of  material  changes. 

Our  theory  has  implications  tor  both  psychology  and  neurophysiology.  In  particular,  Heriny’s 
notion  of  opponent  colors  and  psychologically  unique  primaries,  and  Land’s  results  in 
two  color  projection  can  be  interpreted  as  different  aspects  of  the  visual  system’s  goal  of 
categorizing  materials.  Also,  the  theory  provides  two  basic  interpretations  of  the  function  of 
double  opponent  color  cells  described  by  neurophysiologists. 
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We  argue  that  one  of  the  early  goals  of  color  vision  is  to  distinguish  one  kind  of 
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that  one  region  has  greater  intensity  at  one  wavelength  than  at  another  wavelength,  and 
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arisen  from  distinct  materials  in  the  scene.  We  call  this  material  change  circumstance 
the  "opposite  slope  sign  condition."  With  this  criterion  as  a  foureiation,  we  construct 
a  representation  of  spectral  information  that  facilitates  the  recognition  of  material 
changes. 
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1.  Introduction 

The  human  visual  system  performs  a  remarkable  feat.  The  pattern  of  light  that  reaches 
the  eye  from  a  scene  is  the  result  of  a  complex  interaction  among  several  factors:  the  quality 
of  the  illuminant.  the  geometry  of  the  scene,  and  the  properties  of  the  materials  composing 
the  visible  surfaces.  Yet  somehow  these  confounded  factors  are  mostly  separated  in  our 
perception.  We  see  particular  spatial  arrangements  of  objects.  These  objects  appear 
bounded  by  surfaces  having  properties— color  and  texture— roughly  invariant  over  a  range 
of  conditions  of  geometry  and  illumination  To  compute  invariant  descriptions  of  the  material 
properties  of  surfaces  is  an  important  goal  of  any  visual  system.  Such  material  descriptors 
are  useful  for  object  recognition  and  visual  search. 

It's  commonplace  to  assume  color  vision  has  something  to  do  with  capturing  the 
albedoes  of  surface  materials. '  But  exactly  what  aspect  of  the  albedo  function  would  serve 
a  visual  system  best?  Consider  the  grandiose  goal  of  recovering  a  material's  albedo  as  a 
continuous  function  of  wavelength.  Not  only  is  this  goal  impractical;  it  is  counter  to  the 
aim  of  finding  invariant  descriptors.  With  such  an  over-zealous  representation,  unimportant 
variations  in  a  surface  would  prevent  its  being  recognized  as  a  single  region,  a  patch  of 
one  kind  of  stuff.  The  perception  of  the  world  would  be  shattered  with  spectral  acuity  too 
fine;  one  literally  wouldn't  be  able  to  see  the  forest  for  the  trees. 

Here  we  seek  a  representation  of  material  reflectance  in  which  trivial  surface  variations 
can  be  overlooked  in  order  to  appreciate  important  similarities.*  At  the  same  time,  the 
representation  must  ailow  some  discrimination  among  different  materials.  Below  we  develop 
such  a  categorical  color  space,  based  on  a  theoretical  solution  to  the  problem  of  identifying 
material  changes.  A  trichromatic  system,  it  will  be  shown,  yields  a  two-dimensional  color 
space  in  which  the  axes  will  turn  out  to  represent  boundaries  between  different  materials. 
The  four  quadrants  of  the  two  dimensional  space  represent  material  categories. 


2,  Spectral  Information  at  Edges 

When  two  image  regions  arise  from  different  materials  in  the  scene,  the  transition  from 
one  mateiial  to  another  will  usually  bring  about  an  edge  in  the  image.  Thus  we  restrict  our 
search  for  matt-rial  changes  to  edges.  How  can  we  decide  whether  an  edge  is  due  to  a 
material  change? 

An  edge  in  the  image  will  usually  arise  Irom  a  single  event  or  state  of  affairs  in 
the  three  dimensional  scene  (Marr.  1982).  The  most  common  edge  types  are  shadows. 

The  alt>rd<)  of  a  m, iirn.il  is  a  functm  >  of  wavelength  (>|X).  Wl,h  range  (0,  I),  that  indicates  what 
Ir. action  nf  photons  (i  milted  t  v  some  light  source)  at  each  wavelength  will  be  reflected. 

Wo  are  not  suggesting  any  spectral  information  tie  thrown  away  We  are  merely  exploring  a  single 
problem  Olher  problems  may  (Vi,  re  detailed  s|x‘ttral  information. 
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(A)  Lawful  Change  (B)  Material  Change 


Wavelength 

Figure  1  Graphs  of  image  intensity  versus  wavelength.  Each  curve  represents  the  image  intensity 
measurable  from  one  image  region  A)  Two  graphs  of  same  shape;  a  likely  lawful  change  D)  T/vo 
graphs  ol  different  shape:  a  candidate  for  material  change. 


highlights,  surface  orientation  discontinuities,  and  pigment  density  changes.1  Alternatively, 
an  edge  may  be  due  to  a  material  change,  a  discontinuity  between  two  different  kinds  of 
stuff.'  How  can  a  material  change  edge  be  distinguished  from  other  types  of  edges?  Rubin 
&  Richards  (1982)  attempted  to  answer  this  question.  Edges  which  arise  from  shadows, 
orientation  changes  and  highlights  are  lawful  in  the  sense  that  there  are  equations  that 
describe  how  image  intensities  will  change  across  these  edges.  By  contrast,  material 
changes  are  completely  unpredictable;  they  are  arbitrary  changes,  and  as  such,  can  only 
be  inferred  by  ruling  out,  at  a  given  edge,  the  possibility  of  any  of  the  above  lawful  changes. 

To  infer  material  changes,  we  now  face  the  .awkward  prospect  of  having  to  reject, 
one  by  one,  each  of  the  lawful  changes.  Perhaps  there  is  some  method  of  rejecting  all  of 
those  edges  en  masse.  Fortunately,  there  is  a  simple  ordinal  rule  common  to  all  the  edges 
formed  by  lawful  processes:  if  the  intensity  at  one  wavelength  decreases  across  a  lawful 
edge  (shadows,  highlights,  and  so  on)  then  the  intensity  must  also  decrease  at  all  other 
wavelengths  taken  across  the  same  edge  (Rubin  and  Richards,  1982).  When  this  condition 
is  vio.ated,  we  say  there  is  a  “spectral  crosspoint”  across  the  edge.  Spectral  crosspoints 
imply  material  changes:  a  spectral  crosspoint  is  illustrated  in  cig.  2a.  The  spectral  crosspoint 
is  not  the  only  means  of  discovering  material  changes,  however.  We  will  show  that  a  second 
and  independent  condition  holds  for  each  of  the  lawful  processes— namely  the  preservation 
of  ordinality  of  image  intensity  across  wavelength.  A  violation  of  this  condition  implies  a 
material  change. 


Surface  on.  -.t.ition  change  and  shadow  can  coincide  at  r.n  edge.  but  this  exception  is  unimportant 
to  Ihe  arguments  (hat  follow  See  Mubin  A  Richards.  19' V  footnote  16. 

"Wo  consider  inateria!-,  to  conm'i  of  Mir  spectrally  ni.i'ral  imbedding  material  (eg.  cellulose) 
impregnate)  vath  a  single  pigment  (c  g  .  chlorophyll)  A  matenal  change  is  a  change  in  pigment  type, 
or  a  Change  m  both  pigment  and  embedding  material. 
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3.  The  Opposite  Slope  Sign  Inference 

3.1  The  Lawful  Processes 

Figure  t;i  shows  two  imago  intensity  graphs  of  the  same  shape.  Intuitively,  the  two 
graphs,  ot  similar  shape,  arise  from  measurements  taken  on  either  side  of  a  "lawful"  edge 
type.  Figure  th  shows  two  graphs  of  different  shape.  None  of  the  lawful  edge  types  could 
have  produced  such  a  distortion,  and  intuitively  it  seems  that  a  material  change  edge  is 
the  best  explanation  We  now  must  make  explicit  what  we  mean  by  “same  shape”,  and 
then  show  that  this  definition  of  spectral  shape  remains  invariant  across  edges  created  by 
shadows,  changes  in  surface  orientation,  highlights  or  variations  in  pigment  density— namely 
the  lawful  conditions  we  wish  to  reject  as  material  changes. 


Definition  Two  curves  of  intensity  versus  wavelength  h.^ve  the  same  shape  if  the 
ordinal  relations  of  image  intensity  across  wavelength  are  preserved. 


Thus,  if  /V(X)  and  />  (X)  are  image  intensities  as  functions  of  wavelength  measured  on  both 
sides,  A  and  V,  of  an  edge.  /.\ (X)  and  />- (X)  have  identical  ordinality  if,  for  all  X,  and  XJ( 
/.v(X|)  <  /.V(X2)  if  J  />  (X,)  <  MX2).  Note  that  two  image  intensity  functions  of  identical 

ordinality  will  have  local  extrema  at  the  same  values  of  wavelength. 

i 

! 

Given  this  ordinal  definition  of  “same  shape”,  Appendix  1  shows  that  the  ordinality 
relationship  is  preserved  across  all  edges  arising  from  the  lawful  edge  types,  provided  that 
the  following  two  conditions  hold; 

Gray  world  condition:  The  average  of  all  the  different  albedoes  in  the  scene  will 
be  a  spectrally  flat  "gray",  so  that  the  diffuse  reflected  light  will  have  the  same 
spectral  character  as  the  direct. light. 


Spectral  normalization .  The  spectral  samples  of  image  intensity  have  been 
normalized  with  respect  to  the  color  of  the  illuminant. 

(The  need  for  the  second  condition,  namely  spectral  normalization,  will  be  eliminated 
subsequently.) 


3.2  The  Opposite  Slope  Sign  Operator 

We  now  can  proceed  to  test  for  "same  shape"  using  the  ordinality  relation.  If  ordinality 
is  violated  across  an  edge,  then  we  infer  the  edge  does  not  arise  from  one  of  the  "lawful” 


3 


I 


RUBIN  AND  RICHARDS 


COLOR  AND  MATERIAL  CATEGORIES 


(A) 


tn 

c 

a> 
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<D 

CT» 


£  (0 


Independence  of  Crosspoint  and 
Opposite  Slope  Sign 


>< 


Ai  A  2 


:S- 


(B) 


(D) 


Wavelength 


A|  X2 


Figure  2  Groph3  of  image  intensity  (ordinate)  versus  wavelength  (abscissa).  Two  wavolength 
samples,  X,  and  X,.  are  shown  An  mage  region  yields  two  samples  of  intensity,  one  for  each 
wavelength,  ana  is  represented  by  the  line  segment  connecting  the  two  sample  values,  a)  &  c)  Two 
examples  ot  the  spectral  crosspoint  (Rubin  8  Hicnards.  190?)  a)  8  b)  Two  examples  of  the  opposite 
slope  sign  condition.  This  is  the  minimal  configuration  that  shows  different  ordinalilies.  Note  that 
the  crosspomt  and  opposite  slope  sign  condition  are  completely  independent,  since  they  can  occur 
together  (a),  or  each  can  occur  alone  (b  and  c),  or  neither  can  occur  (d). 


processes  and  hence  must  represent  a  material  change  (provided  also,  of  course,  that  our 
grey  wijrld  condition  is  not  violated).* 

Wnat  is  the  simplest  way  to  seek  violations  of  ordinality?  A  pair  of  spectral  samples 
suffices  '  Let  the  image  intensities  on  both  sides  of  an  edge  be  measured  at  wavelengths 
X,  and  xi> .  If  image  intensity  at  Xt  is  greater  than  that  at  X2  on  one  side  of  the  edge,  then 
the  ordinality  condition  requires  the  same  relationship  hold  on  the  other  side.  So  if  the 
two  sides  of  the  edge  do  not  have  greater  intensity  in  the  same  spectral  sample,  ordinality 
is  violateq;  the  edge  cannot  be  lawful.  (Details  are  given  in  Appendix  1.)  This  condition 


,  r 


r,lt  is  possible  when  the  grey  world  assumption  is  wrong,  material  changes  will  be  inferred  from 
images.  This  is  not  entirely  bad  news;  ;f  human  perception  also  goes  awry  when  the  grey  world 
assumption  is  violated,  then  our  theory  will  become  more  credible  as  an  account  of  biological  visual 
systems. 
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is  called  the  opposite  slope  sign  condition ,G  Examples  are  shown  in  Fig.  2a  and  2b.  The 
"slope"  of  the  opposite  slope  sign  condition  is  the  slope  of  the  graph  of  intensity  versus 
wavelength;  it  is  an  evaluation  of  the  sign  of  the  derivative  of  the  spectral  image  intensity 
function,  jl . 

More  formally,  given  two  regions  .Y  and  Y  across  an  edge  and  intensity  samples  I 
taken  at  two  wavelengths  X,  and  X2.  we  have  the  following  test  for  a  material  change: 

Opposite  Slope  Sign  Condition: 

(lxxa  ~  -  lr\x)  <  0. 

which  may  be  contrasted  with  the  previously  derived  crosspoint  condition  (Rubin  and 
Richards,  1902): 

Spectral  Crosspoint  Condition: 

(/.vx,  -  Iyx,)(Ix\,  -  Iy\%)  <  0. 

i'Jutt:  tiirti  me  specitdi  Ciosspoim  and  ii ie  opposite  siope  sign  cuuumuns  are  compieieiy 
independent.  Figure  2a  shows  the  two  occurring  together.  Each  condition  can  arise  alone, 
as  shown  in  Figs.  2b  and  2c.  Finally  neither  condition  is  necessary,  as  shown  in  Fig.  2d. 

The  two  conditions  are  related  by  a  kind  of  symmetry.  The  spectral  crosspoint  must 
make  two  comparisons  across  an  edge  (one  for  each  wavelength),  and  combine  them 
logically  (both  comparisons  must  work  out  in  the  correct  way).  The  opposite  slope  sign 
condition  must  make  two  comparisons,  one  within  each  image  region,  and  then  combine 
them  logically  across  the  edge. 

To  summarize:  the  spectral  crosspoint — our  original  means  of  finding  material  changes — 

has  been  augmented  by  a  second  and  independent  material  change  condition:  opposite 

slope  sign.  The  opposite  slope  sign  condition  is  the  key  theoretical  result  on  which  we  will 

base  our  spectral  representation  of  material  types.  We  choose  opposite  slope  sign  rather 

than  the  crosspomt.  because  the  opposite  slope  sign  condition  tells  us  something  about 

each  of  the  two  regions  that  produce  it.  Namely,  one  region  has  positive  spectral  slope,  the 

other  negative.  By  contrast,  the  spectral  crosspoint  cannot  be  decomposed  into  assertions 

about  the  two  regions  that  produce  it.  In  a  crosspoint,  spatial  and  spectral  information  are 

’’The  opposite  slope  sign  condition  is  described  here  as  existing  statically,  across  an  edge.  It  is  a 
spatial  comparison  of  spectral  information.  A  comparison  of  spectral  information  in  tune  is  equivalent. 
Such  a  temporal  opposite  slope  s<r;n  condition  would  work  as  follows:  An  eye  could  sweep  across 
an  edge,  and  the  .spor.unt  information  before  and  after  the  movement  could  be  compared.  Similarly, 
there  is  a  temporal  equivalent  of  too  cicsspoint.  Consequences  of  these  isomorphic  computations  in 
the  temporal  domain  will  not  be  exploit'd  here. 
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hopelessly  intertwined.  We  do  not  cast  aside  the  crosspoint,  though.  It  will  play  a  vital  role 
in  correcting  for  the  spectral  content  of  the  illuminant. 

4.0  Spectral  Normalization 

For  the  opposite  slope  sign  tes*  to  find  material  edges  successfully,  it  is  necessary 
for  the  measured  spectral  intensities  to  be  normalized.  That  is,  these  samples  must  be 
transformed  to  what  they  would  have  been  under  a  spectrally  flat  ("white”)  illuminant. 
Clearly  if  no  correction  is  applied,  then  the  stronger  spectral  skew  of  an  illuminant  may 
not  only  reduce  the  number  of  observed  opposite  slope  sign  pairs,  but  more  seriously,  may 
transform  pairs  having  the  same  slope  sign  into  pairs  that  are  seen  as  having  an  opposite 
slope  sign. 

By  contrast,  the  spectral  crosspoint  condition  is  insensitive  to  the  spectral  content 
of  the  illuminant,  as  can  be  seen  by  inspecting  panels  A  and  C  of  Fig.  2.  (See  Rubin 
&  Richards,  1982,  for  a  more  formal  treatment.)  We  capitalize  on  this  property  of  the 
crosspoint  to  devise  a  theory  of  spectral  normalization.  Once  the  image  has  been  spectrally 
normalized,  it  is  as  if  the  illuminant  were  white.  The  opposite  slope  sign  condition  will  now 
be  able  to  find  correctly  a  maximum  number  cf  material  changes. 

Consider  now  a  scene  composed  of  a  large  number  of  randomly  selected  materials.  For 
each  image  region  (simple  closed  curves  defined  by  edges),  take  two  samples  of  intensity 
lx,  and  lx ,  at  wavelengths  X,  and  X2.  Each  region  will  be  associated  with  a  spectral  slope 
sign,  which  is  just  the  sign  of  the  difference  lx,  -  lx f  the  illuminant  were  white  (same 
photon  flux  at  all  wavelengths),  we  would  expect  to  have  roughly  equal  numbers  of  regions 
of  positive  spectral  slope  and  regions  of  negative  spectral  slope.  This  expectation  is  based 
on  two  assumptions.  The  first  is  that  there  is  a  random  collection  of  materials  in  the  scene. 
The  second  is  that  materials  in  the  world  are  such  that  a  random  collection  of  them  will  be 
divided  equally  between  positive  and  negative  spectral  slope. 

As  suggested  above,  normalization  requires  a  collection  of  image  regions  that  arises 
from  a  random  set  of  materials.  What  about  using  all  image  regions?  The  set  of  all  image 
regions  is  not  likely  to  represent  a  random  collection  of  materials,  because  many  materials 
will  recur  in  several  image  regions.  For  example,  if  a  cast  shadow  cuts  across  a  single 
piece  of  material,  that  material  will  be  twice  represented,  once  for  each  side  of  the  shadow 
edge.  A  second  example  arises  with  pigment  density  changes.  In  a  forest  scene,  all 
leaves  are  composed  of  the  same  mateual  (chlorophyll  embedded  in  a  cellulose  base).  A 
sensible  normalization  scheme  would  not  take  each  leaf  as  a  distinct  patch  of  material; 
minor  variations  in  pigment  density  from  leaf  to  leaf  ought  to  be  ignored. 
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It  seems  clear,  then,  that  not  all  image  regions  should  participate  in  normalization. 
Perhaps  a  subset  of  image  regions  can  be  found  that  is  more  likely  to  represent  a  random 
collection  of  materials.  The  spectral  crosspoint  offers  a  means  of  finding  such  a  random 
subset  of  regions.  Suppose  that  instead  of  taking  each  image  region  as  a  distinct  material, 
we  took  only  pairs  of  regions  that  have  a  spectral  crosspoint  on  the  edge  between  them. 
We  would  be  guaranteed  that  each  pair  of  regions  would  correspond  to  distinct  materials. 
The  pairs  of  different  material  regions  found  with  the  crosspoint  will  be  the  subset  of  image 
regions  that  will  be  used  for  normalization. 

Our  normalization  scheme  works  like  this:  Recall  that  we  expect  the  regions  found  by 
the  crosspoint  to  represent  a  random  collection  of  materials.  So  we  expect  roughly  the 
same  number  of  regions  having  positive  spectral  slope  as  negative.  For  the  subset  of  image 
regions  defined  by  the  crosspoint,  tally  the  number  having  positive  spectral  slope  and  the 
number  having  negative  slope.  If  the  numbers  are  approximately  equal,  our  expectation  has 
been  met;  we  can  infer  that  the  illuminant  is  white  (spectrally  flat).7  Suppose  to  the  contrary 
that  the  number  of  regions  of  positive  spectral  slope  exceeds  the  number  of  negative-slope 
regions.  Then  we  can  infer  that  the  illuminant  is  more  intense  at  long  wavelengths  than 
at  short.  (Positive  spectral  slope  means  greater  intensity  in  the  longer  wavelength  sample.) 
Now  multiplicatively  scale  one  of  the  spectral  samples.  In  the  example  here,  we  need  to 
multiply  all  long  wavelength  samples  by  some  number  less  than  one.  Exactly  which  number? 
The  one  that  will  fulfill  our  expectation  of  equal  numbers  of  positive  and  negative  spectral 
slope.  That  is,  multiply  all  long  wavelength  samples  by  some  number  (less  than  one)  such 
that  half  of  the  regions  under  consideration  will  have  greater  intensity  |n  the  modified  long 
wavelength  sample  than  the  short  wavelength  sample,  and  half,  the  reverse.  For  a  large 
number  of  samples,  the  multiplicative  constant  of  normalization  can  be  calculated  from 
the  mean  value  of  the  spectral  slopes  of  all  regions  participating  in  crosspoints.  See  the 
algorithm  for  spectral  normalization  in  Appendix  2. 

This  crosspoint  normalization  scheme  has  some  useful  properties.  Each  image  region 
used  has  the  same  potency  in  normalization,  regardless  of  the  size  of  the  region.  That  is, 
each  pair  of  image  regions  (found  with  the  crosspoint)  maps  to  a  pair  of  data  points,  one 
for  each  region.  This  is  good  for  two  reasons.  First,  the  scheme  is  independent  of  image 
region  areas.  This  is  desirable  since  we  would  not  want  visual  systems  to  treat  an  image  of 
a  large  blue  thing  and  a  small  red  thing  differently  from  an  image  of  a  small  blue  thing  and 

7Note  there  must  be  some  crosspoints  for  normalization  to  proceed.  If  there  are  no  crosspoints, 
there  are  no  regions  to  consider.  So  although  it  is  technically  true  that  there  are  equal  numbers 
of  positive-slope  regions  and  negative-slope  regions  (namely,  zero),  we  do  not  want  to  infer  the 
illuminant  is  white  for  two  reasons.  First,  we  have  no  information  about  any  image  region,  and  thus 
it  seems  imprudent  to  guess  blindly  that  the  light  is  white.  Second  we  have  evidence  that  the  scene 
consists  of  a  single  material  since  it  has  no  crosspoints.  Normalization  would  bring  about  material 
change  assertions  via  the  opposite  slope  sign  condition,  in  contradiction  to  the  evidence  of  uniformity 
from  the  crosspoint. 
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It  is  worth  comparing  our  crosspoint  normalization  with  Land's  latest  normalization 
theory.  Land's  (1903)  scheme  involves  comparing  the  image  intensity  of  a  target  region 
with  that  of  a  few  hundred  random  locations  in  the  image.  In  such  a  theory,  the  larger 
an  image  region,  the  more  random  locations  it  will  contain.  Land’s  theory  is  therefore 
area-based,  while  ours  is  independent  of  the  particular  sizes  of  image  regions.  Our  theory 
makes  different  predictions  from  Land’s:  we  expect  no  effect  on  normalization  from  the  sizes 
of  image  regions,  or  from  the  lengths  of  image  edge  segments. 

5.0  Choosing  a  Representation 

Assume  now  that  the  image  has  been  normalized  using  the  spectral  crosspoint 
condition,  as  described  in  section  4.  We  next  select  a  representation  of  spectral  information 
based  on  that  rule.  In  particular,  we  seek  a  smple,  convenient  spectral  representation  of 
materials  that  is  invariant  under  shadow,  highlight,  surface  orientation  change,  and  pigment 
density  change. 

For  any  region  in  the  image,  intensity  can  be  measured  at  a  long  wavelength  and  at 
a  second,  shorter  wavelength.  Call  these  two  measurements  of  image  intensity  L  and  S, 
respectively,  for  each  image  region.  Suppose  we’d  like  to  represent  the  spectral  character 
of  a  region  with  a  single  number,  namely  some  mapping  of  the  pair  (/.,  5).  Furthermore,  we 
would  like  the  mapping  (/,,  S)  to  be  invariant  under  the  lawful  changes.  The  recognition  of 
material  differences  would  be  easy  in  such  a  representation.  A  single  material  in  its  different 
guises— fully  lit,  shadowed,  having  different  densities  of  pigmentation,  with  different  surface 
orientations— would  map  ideally  to  a  single  point.  If  there  were  such  a  mapping,  then 
whenever  two  image  regions  mapped  to  distinct  points,  we  would  know  they  corresponded 
to  distinct  materials. 

The  lawful  edge  types  are  unfortunately  so  diverse  that  there  is  no  function  giving  us  the 
desired  mapping.  No  single  continuous  function  of  (/,,  S)  will  L>e  invariant  under  multiplicative 
(shadow),  exponential  (pigment  density),  and  additive  (highlight)  changes.  Material  change, 
then,  cannot  be  reduced  to  the  problem  of  distinguishing  two  points  in  the  range  of  some 
function. 

The  problem  isn’t  hopeless,  however,  for  there  is  a  continuous  function  invariant  under 
some  of  the  lawful  changes,  namely  the  multiplicative  ones  (shadow  and  surface  orientation 
change).  Consider  again  the  two  image  intensity  samples  S  and  f..  The  quotient  £  will  have 
the  identical  value  on  both  sides  of  a  surface  orientation  change  or  a  shadow  edge.  The 
simple  quotient  is,  of  course,  not  unique  in  remaining  constant  across  an  orientation  edge. 
Many  functions  of  the  two  samples  I,  and  S  have  the  same  property.  We  will  choose  among 
three  simple  functions  having  this  property: 
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Many  functions  of  the  two  samples  L  and  S  have  the 
three  simple  functions  having  this  property: 

L  l 

S  I,  +  S 

How  can  we  select  among  these  candidates? 
into  the  unbounded  interval  (0,  oo),  while  the  other  two  functions  take  image  intensities  into 
closed  intervals.  ( (  '-s.  maps  intensities  into  [ft,  I];  maps  into  [-1,1].)  The  function  (;  will 
be  rejected,  since  any  reasonable  computational  system  will  be  better  off  using  quantities 
that  fall  within  a  closed  interval,  rather  than  those  that  could  be  arbitrarily  large.  To  choose 
between  the  two  remaining  candidate  functions  we  consider  the  ease  of  discovering  material 
changes  in  these  two  maps.  In  particular,  how  does  the  opposite  slope  sign  condition  appear 
in  each  of  the  candidate  mappings? 

Given  two  image  regions  X  and  Y,  let  F  denote  the  function  j-£s,  so  that  F(X)  and 
F[)  )  are  the  values  of  the  function  F  of  regions  A'  and  V,  respectively.  Then  for  F,  the 
opposite  slope  sign  condition  is  expressed  by  \.n<jn(F(X)~  |)  ^  />i<jn(F(Y)-  £)].  (The  reason 
for  this  expression  is  that  the  function  F  takes  on  the  value  £  whenever  L  =  S.) 

Let  C,  denote  the  function  a  common  measure  of  contrast.  This  is  a  simple 
function  that  facilitates  the  computation  of  material  cnange.  The  sign  of  (7  is  the  sign  cf 
the  spectral  slope  of  an  image  region.  That  is,  (at;/«(f7(A))  ^  .i«</n(<7(V'))]  emerges  as  the 
opposite  slope  (material  change)  condition. 

We  prefer  the  function  G  to  the  F  for  our  representation.  Whereas  to  determine 
material  change  with  G  requires  only  a  sign  check,  with  F,  the  system  must  maintain  the 
constant  i  and  perform  two  subtractions.  The  particular  choice  of  F  or  G,  though,  seems 
not  to  be  critical  for  the  goals  we  have  in  mind. 

Figure  3  shows  the  interval  [-1,  t],  the  range  of  the  function  G.  Two  image  regions 
corresponding  to  lit  and  shadowed  versions  of  the  same  material,  or  two  different  surface 
orientations,  will,  by  design  of  G,  be  mapped  to  the  same  point.  This  is  shown  in  Fig.  3a. 
Two  image  regions  of  different  pigment  density  have  the  same  slope  sign;  hence,  in  the  G 
map,  the  corresponding  pair  of  points  cannot  straddle  the  zero.  The  same  holds  for  a  pair 
of  points  corresponding  to  a  highlight  and  a  neighboring  matte  region.  The  latter  two  edge 
types  are  shown  in  the  G  mapping  in  Fig.  3b.  If  two  image  regions  are  mapped  to  points 
straddling  the  zero  (Fig.  3c),  they  arise  from  different  materials. 

To  summarize,  we  sought  a  function  of  spectral  information  invariant  over  the  lawlul 
changes.  That  goal  being  impossible,  we  chose  for  two  reasons.  First,  it  is  invariant 
across  shadows  and  surface  orientation  changes.  Second,  finding  material  changes  with 
the  opposite  slope  sign  condition  is  easy.  The  range  of  the  function  can  be  divided  into 


same  property.  We  will  choose  among 


L_-S 
1,  +  S 


(7) 


The  function  takes  image  regions 
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How  Edges  Map  into  the 
Spectral  Representation 


(A)  .[ 


Shadowed 

region 


V 


Lit 

region 


J 


I 


ki 

L+S 


Pigment  density  change 

i — J—X — i - 1 

-I  0  I 


L 

-I 


Materia!  change 

/  ,  V 


J 

I 


Ficure  3  Kow  various  processes  appear  in  the  spectral  representation  implied  by  the  mapping 
the  range  of  which  is  1-1,1  J.  a)  Two  image  regions  differing  only  in  surface  orientation  or  shadow  map 
to  a  single  point,  b)  Two  regions  differing  as  matte  and  highlighted,  or  as  two  different  degrees  of 
pigmentation  density,  map  to  the  same  half  of  the  range,  i.e.,  they  map  to  point3  having  same-sign 
coordinates,  c)  Only  two  different  materials  can  map  to  points  straddling  the  zero,  i.e.,  to  points  of 
different-sign  coordinates. 


two  parts,  (-1,0)  and  (o,  l).  Materials  with  albedoes  of  positive  spectral  slope  sign  will  map 
into  the  positive  half  of  the  range,  and  negative-sloping  albedoes  to  the  negative  part  of  the 
range.8 

Finally,  it's  worth  reiterating  why  we  built  our  spectral  representation  around  tho 
opposite  slope  sign  condition,  and  not  the  spectral  crosspoint.  Spectral  slope  sign  is  an 
invariant  property  of  a  material's  albedo  function.*  The  opposite  slope  sign  condition  can 
be  decomposed  into  separate  meaningful  statements  about  properties  of  two  image  regions: 
The  slope  sign  of  one  region  is  positive,  and  that  of  the  other,  negative.  We  know  something 
about  each  region.  The  crosspoint,  by  contrast,  hopelessly  confounds  spatial  and  spectral 
information.  Higher  goals  of  color  vision  involve  describing  the  properties  of  individual 
image  regions,  and  cannot  be  reached  by  the  crosspoint  alone. 


"Many  continuous  maps  share  the  same  invariance.  Wo  selected  our  map  on  the  basis  of  algorithmic 
considerations.  The  particular  choice  is  independent  of  the  theory  of  finding  material  change  edges. 
'’Since  a  material  is  defined  as  a  kind  of  stuff,  a  single  material  can  have  different  albedoes  as  pigment 
density  changes.  What  stays  constant  over  these  changes  in  density  of  pigment  is  spectral  slope  sign. 
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Trichromatic  Representations 


Figu  re  4  Steps  in  the  construction  of  the  trichromatic  material  representation,  a)  Two  axes  comparing 
I.  and  v/,  and  .S'  and  M  samples,  are  joined  orthogonally.  Each  quadrant  is  a  material  category. 
Points  in  dilforent  quadrants  correspond  to  distinct  materials.  Points  within  one  quadrant  may  belong 
to  the  same  material;  they  are  considered  equivalent  in  this  representation,  b)  Tl.e  line  of  unit  slope 
in  the  figure  above  represents  the  comparison  between  S  and  I.  samples.  Aoding  the  unit  slope  line 
divides  the  color  space  into  six  regions  or  "hextants.”  Points  in  different  hextants  arise  Irom  different 
materials.  Note  the  hextants  do  not  have  equal  areas. 


6.0  Trichromacy:  Finding  More  Material  Changes 

Suppose  we  add  a  third  spectral  sample,  call  it  M,  to  our  original  S  and  L  samples. 
Adding  a  third  spectral  sample  will  allow  the  detection  of  new  kinds  of  material  changes.10 
However,  more  importantly,  the  number  of  basic  material  categories  will  be  increased  from 
two  to  six. 

In  the  two-wavelength-sample  material  representation,  an  image  region  is  encoded 
essentially  by  the  rank  order  of  the  spectral  samples,  or  equivalently  by  the  sign  of  the  slope 
of  the  line  segment  connecting  the  samples.  Thus,  given  two  wavelength  measurements, 
there  are  two  types  of  material— negative  slope  and  positive,  slope.  With  three  wavelength 
samples,  an  image  region  is  associated  with  three  slope  signs— a  slope  between  each  pair 
of  samples  (SM,  Ml,,  St,).  There  are  six  possible  rank  orderings  of  the  measurements 
(3!  —  6),  and  thus  six  possible  basic  material  types.  Any  two  regions  that  produce  distinct 
rank  orderings  of  the  wavelength  samples  will  bring  about  me  or  more  opposite  slope  signs. 
Any  two  such  regions  must  therefore  be  distinct  material?;. 

As  a  first  step  in  constructing  the  trichromatic  material  representation,  we  combine 
slope  information  from  two  of  the  three  pairs  of  samples.  Arbitrarily,  we  begin  with  S M 

'"The  additional  number  of  ma*erial  changes  detected  with  each  new  spectral  sample  will  drop 
sharply  after  the  third  sample.  The  reason  is  that  the  albedoes  of  natural  objects  (in  the  visible  range) 
are  typically  slow-changing  functions  of  wavelength  (Krinov,  1971;  Snodderly,  1979).  Cohen  (1964) 
showed  that  three  carefully  chosen  (unctions  ot  wavelength  captured  over  99%  of  the  albedo  functions 
of  Munsell  chips. 
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and  l.M ,  combining  the  information  in  a  two  dimensional  space  as  shown  in  Fig.  4a.  Image 
regions  are  mapped  to  points  in  the  square  [-1,1]  X  [  I,  l],  and  a  pair  of  points  separated 
by  an  axis  (or  both  axes)  correspond  to  two  regions  of  different  material,  just  as  did  a  pair  of 
points  straddling  the  zero  in  Fig.  3c,  Any  pair  of  points  in  a  single  quadrant  may  arise  from  a 
single  material.  This  is  the  sense  in  which  quadrants  represent  material  categories.  Without 
yet  considering  comparisons  between  .s'  and  /.  samples,  we  already  have  a  categorical 
representation  in  Fig.  4a.  in  which  in  each  quadrant  corresponds  to  a  material  category. 

Let's  now  examine  the  third  pairing  oi  samples,  s’  and  What  condit:on  holding 
between  a  pair  of  points  in  the  preliminary  representation  of  Fig.  4a  corresponds  to  the 
opposite  slope  sign  condition  between  .s’  and  1.7  It  is  easily  shown  that  if  a  pair  of 
points  straddles  the  line  of  unit  slope,  the  points  arise  from  materials  with  opposite  {.S’  and 
/,)  slopes."  Furthermore,  not  just  the  sign,  but  the  continuous  value  of  the  L  to  S 
comparison  is  contained  implicitly  in  the  representation  defined  by  ordered  pairs  £rw) 
that  Fig.  4a  illustrates.11 

The  unit  slope  line  in  the  SM-I.M  space  therefore  has  special  significance,  and  is 
added  to  the  representation  as  a  third  material  change  axis  in  Fig.  4b.  A  pair  of  points  lying 
across  any  of  the  three  axes  will  correspond  to  distinct  materials  Thus,  each  of  the  six 
sectors  of  Fig  4b  corresponds  to  a  material  type,  or  equivalently,  to  a  rank  ordering  of  the 
three  samples  The  particular  rank  ordering  associated  with  each  ’’hextant"  is  shown  in  Fig. 
4b  Note  the  hextants  of  Fig  4b  do  not  have  equal  areas  The  original  pair  of  axes  can  be 
lomed  in  a  skew  fashion  to  allocate  more  or  less  area  to  the  different  material  categories. 

To  summarize,  image  intensities  are  measured  at  .s’.  M  and  /,,  normalized  according 
to  the  crosspoint  normalization  of  section  4,  and  mapped  to  ({--,•$),  ;';  w)  'n  a  rectangular 
coordinate  system,  initially  creating  four  basic  material  types  A  further  subdivision  into  six 
types  can  arise  by  using  the  line  of  unit  slope  as  a  third  axis,  dividing  the  region  [ -1,1]* 
into  six  regions,  each  corresponding  to  a  different  material  type  Points  in  different  hexta.its 
arise  from  different  materials,  whereas  points  common  to  one  hextant  may  arise  from  lawful 
edge  events  occurring  on  a  single  material. 

Algorithm  aficionadvjs  should  turn  to  Appendix  2,  where  we  sketch  a  procedure  for 
spectral  categorization  based  on  the  above  theory. 


"The  line  o»  unit  s'ope  is  'ivenby  *,  JJ  this  is  equivalent  to  (>’  \r|(/..Ar)  .  (S  *  M){L  -  JW), 

or  N  /.  Points  above  i.hs  unit  slope  Imp  coi  respond  to  /.  >  points  below  to  S  >  h. 

Gwen  the  values  ( » .  JJ.  ] .  ^  I-  we  can  compute  the  value  of  '  '  Let  O  ' "  fj  and  It  -  S 

Thpo  '  >  <v  a  “ 

Inen  i:m  v'i  i- 
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7.0  Relation  to  Psychophysics  and  Neurophysiology 

Our  spectral  repiesentation  of  material  types  is  but  an  abstract  model  of  biological 
color  vision.  In  our  theory,  certain  terms  are  left  undefined.  We  haven't  described  whot  the 
"spectral  samples"  of  the  theory  are.  and  we  haven't  said  anything  abc  ,t  how  materials  are 
encoded.  How  then  can  we  assess  its  relevance?  Two  linking  assumptions  will  giade  the 
interpretation  of  our  theory.  First,  in  the  discussion  of  the  psychology  of  color  visirn,  we 
will  argue  that  of  the  traditional  color  variables  hue.  saturation,  and  lightness,  it  is  hue  that 
encodes  material  type  Second,  in  the  discussion  of  neurophysiology,  we  ta^-e  the  small  step 
to  identify  the  spectral  samples  of  our  theory  with  the  relative  stimulation  of  the  three  human 
cone  photopigments  (or  combinahons  thereof)  Given  this  interpretation  of  our  theory,  it 
turns  out  that  rioubie  opponent  units  found  in  color  neurophysiology  can  be  understood  as 
performing  the  spectral  crosspomt  and/or  the  opposite  slope  sign  computation. 

7. 1  Psychologically  Unique  Primaries 

Ewald  Hering  ( 1 06*1)  offered  a  psychological  account  of  human  color  perception  that 
was  based  on  the  notion  of  opponent  processes  He  observed  that  "redness  and  greenness, 
or  yellowness  and  blueness  are  never  simultaneously  evident  in  any  colcr,  but  rather  appear 
to  be  mutu.Jly  exclusive."  This  is  a  clear  case  of  categorical  perception.  Reddish  and 
greenish  are  mutually  exclusive  hue  categories,  and  if  hue  is  encoding  material  properties, 
then  the  two  categories  will  partition  materials.  See  Fig.  5a.  Similarly,  bluish  and  yellowish 
will  partition  materials  See  Fig  5b  These  two  sets  of  mutually  exclusive  hue  pairs  divide 
the  color  space  into  four  regions,  as  in  Fig  5c,  just  as  did  our  trichromatic  color  space  (Fig. 
4a). 

Our  claim  that  Hering  s  color  quadrants  correspond  to  our  material  categories  is 
predictive,  we  expect  that  shadows,  surface  orientation  changes,  and  pigment  density 
changes  would  only  rarely  cause  perceived  hue  to  change  from  reddish  to  greenish  (or  vice 
versa)  or  from  yellowish  to  bluish  (or  vice  versa).  As  noted  in  Appendix  I,  highlights  could 

be  troublesome 

The  fact  that  there  are  four  hue  categories  supports  the  idea  that  trichromatic  human 

vision  uses  tv.-o  opposite  slope  sign  checks,  as  in  Fig.  4a,  but  not  the  third,  as  shown  in 

Fig  4b  (Goethe  [1B08|.  however,  proposed  a  theory  of  color  perception  based  on  six 

hue  categories,  which  might  correspond  to  the  use  of  all  three  opposite  slope  sign  checks.) 

'  'Our  theory  ol  crosspoints  and  opposite  slope  signs  was  based  on  spectral  samples  at  a  single 
wavelength  Biological  measurements  ot  the  spectrum  are  broadband  It  turns  out  that  broadband 
samples  cannot  introduce  crosspomts  that  are  false  targets.  That  is.  a  spectral  crc^pomt  found  with 
broadband  samples  is  still  a  reliable  indicator  of  material  change  (Rubin  &  Richards,  19H?,  Appendix 
IV)  The  opposite  slope  sign  condition  may  not  be  as  robust;  more  work  is  needed  to  study  the  effects 
of  broadband  sampling. 
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Opponent  Color  Theory 


Unique 

Green 


Figure  5  (ipriry's  nobon  of  opponent  color  processes  a)  All  colors  .ire  either  reddish  or  greenish, 
but  never  both  b;  An  colors  are  either  bluish  or  yellowish  but  never  both  c)  The  two  pairs  of  mutually 
exclusive  coin's  d.vde  the  ce  or  circle  into  four  quadrants,  similar  to  the  trichromatic  representation 
Ihjt  >ve  devemp  m  fig  6a 


Evidence  from  infants  (Bernstein  ct  al ,  1076)  supports  Bering's  theory  of  four  hue  categories 
as  independent  of  language  and  culture  Pigeons  also  have  categorical  color  perception 
(Wright  A  Gumming,  1971),  suggesting  the  computational  scheme  that  we  propose  here  i3 
fundamental  to  color  vision  across  species. 

Herne's  notion  of  opponent  color  processes  implies  four  special  hues.  They  are 
indicated  in  Fig  5c  These  hues,  which  Hering  called  psychological  primaries,  are  the 

i 

boundaries  that  separate  color  categories.  Primary  red  is  that  hue  among  the  reddish  hues 
that  separates  the  yellowish  from  the  bluish;  primary  blue  is  that  hue  among  the  bluish 
that  splits  the  reddish  from  the  greenish;  and  so  on.  These  primary  colors  are  unstable 
in  the  sense  that  any  deviation  from  them  involves  a  change  of  color  categories.  Hering’s 
psychological  primaries  correspond  to  the  axes  of  our  trichromatic  representation  (Fig.  4a). 


Just  why  these  primaries  have  their  particular  locations  in  the  spectrum  is  an  interesting 
evolutionary  question  not  addressed  here.  One  possibility  is  that  a  creature's  material 
boundaries  are  positioned  in  some  way  as  to  make  the  greatest  number  of  discriminations 
among  materials  encountered  in  its  environment.11  Interesting  work  has  been  done  along 
these  lines  Snodderly  (1979)  attempted  to  relate  the  color  vision  of  New  World  monkeys  to 
the  spectral  characteristics  of  their  jungle  habitat.  Levine  &  MacNichol  (1982)  and  McFarland 


'  'Material  boundaries  can  be  changed  in  two  ways.  The  wavelength  at  which  a  photopigment  captures 
(be  greatest  percentage  oi  photons  can  be  altered,  or  new  "channels"  can  be  created  by  combining 
photopigments.  One  sort  of  combination  of  two  spectral  samples  .S'  and  /,  is  a  rotation;  that  is,  now 
coordinates  {.s'  o  Osin  o,  .s'  sin  o  *-  /,hmi)|  can  be  created  for  some  angle  of  rotation  0.  The  original 
and  rotated  coordinate  systems  will  not  always  agree  about  whether  two  image  regions  satisfy  the 
opposite  slope  sign  condition.  That  is,  the  two  spectral  coordinate  systems  (filtering  only  by  a  rotation 
will  make  different  material  distinctions.  An  angle  0  can  therefore  be  selected  to  maximize  the  number 
of  material  changes  detected 
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A  Munj  (19’5b)  linked  the  phutopigment  characteristics  of  fishes  to  the  spectral  character 
of  tirjht  m  their  environments. 

In  sum,  our  spectral  representation  of  material  categories  is  a  two-dimensional  space 
in  which  each  quadrant  represents  a  material  type,  and  the  axes  represent  the  boundaries 
between  categories  Image  regions  ttiat  map  to  different  quadrants  necessarily  arise  from 
distinct  materials;  image  regions  that  map  to  the  same  quadrant  may  arise  from  a  single 
material  Supposing  that  hue  encodes  material  information,  Hering's  observation  about 
human  color  vision  makes  sense:  hues  are  divided  into  four  fundamental  categories  by  the 
mutually  exclusive  pairs  red-green  and  blue-yellow. 

7.2  Land's  Experiments 

7.2. 1  Two-Color  Projection 

Edwin  Land  (1959a,b)  conducted  some  remarkable  experiments  in  two-color  projection 
of  natural  images.  Some  of  the  phenomena  he  reported  can  be  understood  in  terms  of  our 
"materialistic”  theory  of  categorical  color  vision. 

Land's  paradigm  was  as  follows.  Two  different  black-and-white  transparencies  were 
made  of  a  colorful  natural  scene  by  means  of  long-  and  short-wavelength  filters.15  The  two 
transparencies  were  called  the  long  and  short  records,  respectively.  Corresponding  regions 
of  the  two  records,  in  general,  were  of  different  grey  values.  The  two  records  were  projected 
on  a  screen  in  register,  the  short  record  with  short  wavelength  light,  the  long  record  with 
long  wavelength  light.  Surprisingly,  the  resulting  image  was  richly  colored  and  faithful  to 
the  original  still-life. 

Land’s  (1959a, b)  work  was  basically  descriptive.  He  found  a  means  of  predicting  the 
hue  name  of  a  region  in  the  two-color  reconstruction.  The  intensity  of  long-wavelength 
light  in  the  region  was  expressed  as  a  fraction  of  the  maximum  long-wavelength  intensity 
in  the  entire  image.  The  same  was  done  for  short-wavelength  intensity,  yielding  a  pair  of 
numbers  (each  between  0  and  1).  This  pair  of  numbers  (fraction  of  maximum  .s’,  fraction  of 
maximum  /,),  plotted  on  log-log  axes,  yielded  a  coordinate  system  that  Land  used  to  relate 
image  intensity  to  perceived  hue.  Land’s  coordinate  system  (hereafter  called  "Landspace") 
is  shown  in  Fig.  6a. 

We  will  now  try  to  relate  our  current  work  to  Land’s  findings.  Whereas  Land  began  with 
some  surprising  experimental  observations  of  color  appearance,  we  took  image  intensity 

'"The  transparencies  did  not  consist  solely  of  black  regions  and  white  regions,  but  rather  the  lull 
range  ot  grey  values  between  black  and  white. 
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equations  as  the  starting  point  of  our  theoretical  investigation  of  the  problem  of  discriminating 
materials  We  will  show  how  these  two  approaches  dovetail. 

Our  argument  below  consists  of  four  maior  points  First,  we  look  at  how  the  special 
crosspoint  appears  in  Landspace.  Second,  we  propose  that  an  absence  of  crosspoints 
should  cause  a  total  failure  of  Landspace,  and  note  the  failure  conditions  already  observed 
for  Landspace  correspond  to  such  an  absence  Accordingly,  we  make  some  predictions 
for  two-color  projection  that  conflict  with  predictions  in  the  literature.  Third,  we  note 
the  opposite  slope  sign  condition  is  identical  to  the  fundamental  split  between  wa  n  and 
cool  color  categories  in  Landspace.  Finally,  we  suggest  a  straightforward  extension  of 
our  crosspoint  normalization  theory  that  would  account  for  a  peculiar  result  in  two-color 
projection. 


7.2.2  The  Spectral  Crosspoint  in  Landspace 

In  general,  the  light  source  for  a  scene  will  not  be  white.  (A  white  source  is  one  that 
emits  the  same  flux  of  photons  at  each  wavelength.)  Suppose  we  take  two  spectral  samples 
of  image  intensity  S  and  L  Spectral  normalization  is  any  procedure  that  transforms  S  and 
I,  into  new  values  S'  and  //,  where  the  latter  measurements  would  have  been  obtained  had 
the  illuminant  been  white. 

Land's  normalization  is  ( S'.L ')  =  (s£— ,  j— 7),  where  Sma *  and  are  the  greatest 
intensities  measured  in  the  S  and  L  samples  throughout  the  image. 

Our  theory  of  normalization  is  based  on  the  spectral  crosspoint,  as  discussed  in  section 
4.  To  relate  our  current  work  to  Land’s  experiments,  we  must  ask  how  spectral  crosspoints 
appear  in  Landspace.  We  claim  that  a  crosspoint  corresponds  to  a  pair  of  points  in 
Landspace  that  form  a  line  segment  of  negative  slope  (in  Landspace).  To  avoid  confusion, 
we  will  refer  to  the  slope  of  line  segments  in  Landspace  as  "Landslope,"  as  distinguished 
from  spectral  slope  in  plots  of  intensity  versus  wavelength  as  discussed  earlier  in  the  paper. 
(Landslope,  then,  is  a  function  of  a  pair  of  regions,  whereas  spectral  slope  is  a  property  of 
a  single  region.)  Our  claim,  again,  is  that  a  spectra!  crosspoint  corresponds  in  Landspace 
to  a  pair  of  points  of  negative  Landslope.  The  proof  follows. 

Suppose  there  is  a  crosspoint  between  regions  .V  and  Y.  Then,  say,  Sx  >  Sr  and 
/-.v  <  /->■•  Does  the  crosspoint  imply  some  sort  of  relationship  among  the  Landspace 
coordinates  for  X  and  (S\-,  l.’x)  and  (.s’j ,  /,*)?  It’s  easy  to  see  from  the  definition  of 
Landspace  coordinates  that  S\  >  Sy  and  t.\  <  /.; .  Now  Landslope  is  given  by 
so  the  Landslope  of  crosspoint  regions  X  and  Y  is  negative.  (Note  the  assignment  of  S'  to 
the  abscissa  is  irrelevant  to  the  result.) 


/ 


\ 
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Landspace  and  Some  Achromatic  Loci 


Figure  6  Landspace  and  some  of  its  achromatic  loci,  as  discovered  by  Land  (1059a).  A)  Land’s 
coordinate  system  (adapted  from  Fig.  1  of  Land,  1959b)  that  relates  perceived  hue  to  the  fraction  of 
maximum  long-  and  short-wavelength  light  (expressed  on  log-log  axes).  This  coordinate  system  we 
call  "Landspace."  D)  Image  regions  correspond  to  a  line  of  unit  Landslope.  Such  an  image  (as  well 
as  the  next  two)  results  in  a  monochromatic  percept,  (This  is  produced  by  placing  identical  records 
in  the  long-  and  short-wavelength  piojectors.)  C)  A  line  of  zero  slope.  (This  is  created  by  removing 
the  record  Irom  the  long-wavelength  protector,)  D)  A  line  of  slope  - 1 .  (One  record  is  placed  in  the 
short-wavelength  projector,  and  its  photographic  negative  is  placed  in  the  long-wavelength  projector.) 

7.2.3  Failures  ot  Landspace 

Landspace  is  a  way  of  predicting  the  perceived  hue  of  a  region  given  the  ratio  of  its 
intensity  to  the  maximum  intensity,  at  long  and  short  wavelengths.  This  predictive  scheme 
is  successful  for  two-color  projection  of  natural  images.  Land  noticed,  however,  that  for 
certain  contrived  images,  his  coordinate  system  failed  totally.  These  images  were  seen 
as  achromatic  (or  monochromatic).  What  did  these  concocted  failure  conditions  have  in 
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common?  Land  (1959a)  suspected  that  "any  arrangement  which  yielded  points  falling  on 
a  straight  line  (in  Landspace],  or  even  on  a  simple  smooth  curve,  would  be  colorless." 
(Judd  (i960)  formalized  Land’s  results  on  failure  conditions  ]  We  will  show  that  the  failure 
conditions  Land  has  discovered  correspond  to  situations  in  which  our  theory  is  unable  to 
make  any  material  distinctions.  Furthermore,  we  will  show  our  theory  predicts  stricter  failure 
conditions  than  does  Land  in  his  conjecture.1* 

Figures  Sb.c.d  depict  three  concocted  situations  that  Land  found  (and  Judd  (1960) 
verified]  to  cause  a  breakdown  of  Landspace.  In  figures  6b  and  6c,  the  failure  loci  are 
straight  lines  of  non-negative  Landslope.  Notice  that  for  such  loci,  there  can  be  no  spectral 
crosspoints,  since  crosspoints  correspond  to  point-pairs  of  negative  Landslope. 

Our  normalization  scheme,  re-cast  in  Landspace,  calls  for  inspection  of  all  point-pairs 
of  negative  Landslope,  sir.ce  this  subset  of  points  is  more  likely  to  arise  from  a  random  set  of 
materials  than  the  totality  of  points.  So  a  visual  system  using  our  normalization  procedure, 
finding  no  point-pairs  of  negative  Landslope  (no  crosspoints),  would  fairly  conclude  that 
there  are  no  material  changes  and  hence  only  a  single  material  is  present.  A  monochromatic 
(or  achromatic)  percept  is  an  apt  result,  then,  for  a  system  that  encodes  material  type  by 
hue. 

Consider  next  a  collinear  collection  of  points  of  negative  Landslope  in  Landspace, 
Normalization  can  proceed  according  to  our  scheme,  since  spectral  crosspoints  are  avail¬ 
able.  Thus  we  disagree  with  Land's  (1959a)  conjecture  that  all  collinear  sets  of  points  will 
be  failures.  We  predict  the  locus  shown  in  Fig.  7a  will  yield  a  range  of  hues.  Only  collinear 
sets  of  positive  Landslope  will  fail. 

There  is  one  special  exception  to  our  prediction  that  collinear  loci  of  negative  Landslope 
will  produce  chromatic  percepts.  A  set  of  points  of  Landslope  -1  (Fig.  6d)  corresponds  to 
an  isoluminance  image.  Such  an  image  has  no  luminance  edges,  and  has  long  been  known 
to  disrupt  vision  (Evans,  1948).  We  have  argued  elsewhere  (Rubin  &  Richards,  1982)  that 
crosspoints  are  only  meaningful  across  edges,  and  hence  should  only  be  sought  across 
luminance  discontinuities.  Thus  the  isoluminance  condition  (the  locus  of  Landslope  -1) 
implies  an  absence  of  crosspoints  and  a  failure  of  normalization,  leading  to  an  achromatic 
percept.  \ 

We  turn  next  to  Land's  conjecture  that  curved  loci  in  Landspace  will  yield  achromatic 

percepts.17  We  believe  this  is  an  overgeneralization.  We  predict,  along  with  Land,  that  the 

"'Land’s  conjecture  (that  smooth  one-d'mensional  loci  in  Landspace  w;ll  be  seen  as  achromatic) 
is  problematic.  It  seems  difficult  to  legislate  whether  a  collection  of  points  in  a  plane  constitutes  a 
curvilinear  arrangement  or  defines  an  area.  A  smooth  curve  can  be  drawn  through  any  collection  of 
points  in  a  plane. 

1  A  line  in  Landspace  does  not  have  absolute  significance  anyway,  since  linehood  depends  on  the 
choice  of  axes.  For  example,  a  line  in  Landspace  with  log-log  axes  will  not  be  a  line  with  linoar  or 
power  axes. 
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(B)  Failure 
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Figu.e  7  Predictions  of  our  theory  that  conflict  with  Land'*  conject  re  that  all  one-dimensional  loci 
in  Lands  pace  will  yield  achromatic  (monochromatic)  percepts.  A)  A  linear  locus  of  negative  Lands!  ope 
(jA  - 1 ).  B)  A  smooth  locus  of  points  without  point-pairs  of  negative  Landsiope  should  be  a  failure 
condition.  C)  A  smooth  locus  of  points  that  has  point-pairs  of  negative  Landsiope  should  produce  a 
range  of  hues. 

non-linear  locus  of  points  in  Landspace  shown  in  Fig.  7b,  since  it  contains  no  point  -pairs  of 
negative  Landsiope  (no  crosspoints),  will  be  achromatic.  In  contrast,  the  one-dimensional 
locus  of  Fig.  7c  has  point-pairs  of  negative  Landsiope,  and  should  yield  a  range  of  hues. 

To  sum  up,  we  have  suggested  that  failure  conditions  of  two-color  projection  occur 
when  there  are  no  spectral  crosspoints.  That  is,  Land  failures  should  c  ~ur  if  there  are  no 
(or  too  few)  point-pairs  of  negative  Landsiope.  Our  predicted  range  of  failure  conditions  is 
therefore  narrower  than  Land's.  Furthermore,  Land's  account  of  failures  is  purely  descriptive; 
ours  is  explanatory  (via  the  theory  of  material  changes). 


1  7, P.4  Opposite  Slope  Sign  and  Landspace 

We  have  argued  that  the  opposite  slope  sign  condition  (between  two  regions)  is  strong 
grounds  for  inferring  the  two  regions  are  composed  of  different  materials.  Can  this  condition 
be  recast  in  Landspace?  The  answer  is  yes:  fwc  regions  in  the  opposite  slope  sign  relation 
map  to  two  points  straddling  the  line  of  unit  Landsiope  in  Landspace ." 

The  argument  is  as  follows.  Image  regions  X  and  Y  satisfy  the  opposite  slope  sign 


'"Land's  early  work  relied  on  two  spectral  samples.  Thus  there  is  only  one  opposite  slope  sign 
condition  to  worry  about,  as  shown  in  Fig.  5.  Our  trichromatic  theory,  sketched  in  Fig.  8,  is  not 
app'icable  to  Land’s  work. 
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condition  if  normalized  image  intensities'”  obey  the  following:  L\  >  S‘x  and  LY  <  SY, 
where  l.\  denotes  the  normalized  intensity  in  the  longwave  sample  o:  region  .V,  and  so 
on.  But  the  last  condition  indicates  that  (>‘v./,*v)  lies  above  the  line  of  unit  Landslope 
in  Landspace  (given  the  abscissa  marks  .s’*  values),  and  {SY,I.Y)  lies  below.  If  the  Land 
normalization  is  correct,  then  we  have  shown  that  two  regions  in  an  opposite  slope  sign 
condition  map  to  a  pair  of  points  in  Landspace  straddling  the  line  of  unit  Landslope.  (For 
many  complex  natural  images.  Land's  normalization  scheme  and  ours  could  yield  similar 
>•650115.  That  is  why  we  can  accept  Land's  scheme  as  approximately  correct.) 

Examine  again  Land's  results  shown  in  Fig.  6a.  Land  observed  that  the  hues  appearing 
above  the  line  of  unit  Landslope  are  all  "warm,”  and  those  falling  below  are  "cool."  (Wilson 
&  Brocklebank  [I960],  in  a  study  of  two-color  projection  phenomena,  noted  that  although 
hue,  saturation,  and  lightness  were  not  precisely  preserved  in  the  two-color  reconstruction 
of  the  original  still-life,  at  least  the  warm/cool  aspect  of  hue  was  invariant.)  The  distinction 
between  warm  and  cool  colors  is  certainly  the  most  fundamental  fact  of  categorical  hue 
perception.  To  sum  up,  given  that  Land's  normalization  has  been  successful,  different 
materials  (as  discovered  by  the  opposite  slope  sign  criterion)  map  in  Landspace  to  two 
points  straddling  the  line  of  unit  Landslope  (and  vice  versa).  In  turn,  two  points  straddling 
the  unit  slope  line  correspond  to  two  qualitatively  distinct  hues,  one  warm  and  one  cool. 
This  observation  suppoits  our  claim  that  hue  is  encoding  information  about  differences  in 
material. 


7.2.5  Doubling  the  Record 

Land  discovered  that  if  he  modified  the  two-color  paradigm  by  placing  a  second 
long-wave  record,  say,  in  the  long-wave  projector,  perception  is  not  substantially  changed. 
How  does  this  transformation  alter  Landspace  coordinates?  The  longwave  coordinates  of 
Landspace  are  squared  (and  hence  reduced  since  Landspace  coordinates  are  between  zero 
and  unity).  The  shortwave  coordinates  are  unchanged. 

Notice  that  to  the  extent  that  this  "doubling  the  record”  manipulation  leaves  perceived 
hue  unchanged,  Landspace  has  failed.  Landspace  was  intended  to  allow  predictions  of 
perceived  hue  for  a  given  pair  of  filters.  But  a  successful  prediction  for  the  normal  two-color 
set-up  will  be  unsuccessful  for  the  doubled  record.  For  example,  in  the  normal  set-up,  the 
line  of  greys  is  the  line  of  unit  s'upe;  when  the  record  is  doubled  and  perception  remains 
the  same,  the  line  of  greys  shifts  to  some  curved  locus  (or  a  line  of  Landslope  two  on 
log-log  axes).  The  perception  of  hue  therefore  depends  not  only  on  the  two  filters,  but  the 
distribution  of  values  in  l  andspace  as  well. 

1  “Recall  that  in  order  for  the  opposite  slope  sign  to  reliably  indicate  material  changes,  the  illuminant 
must  be  white,  or  equivalently,  the. image  must  be  normalized. 
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How  does  the  doubling-the-record  transformation  affect  our  normalization  scheme? 
Crosspoints  are  preserved,  since  doubling  the  record  preserves  ordinal  relations  among  the 
vV  measurements  and  among  the  /,. 

Recall  that  the  premise  of  our  normalization  theory  was  that  given  a  random  collection 
of  materials  under  white  light,  the  median  spectral  slope  would  be  zero.  Suppose  we  divided 
the  random  collection  into  two  subgroups,  one  of  light  materials  (high  average  albedo  over 
wavelength)  and  one  of  dark.  It  seems  that  our  fundamental  premise  would  also  be  true 
of  each  of  the  subgroups.  That  is,  it  seems  reasonable  to  guess  that  the  median  spectral 
slope  of  a  random  assortment  of  dark  materials  would  be  zero,  and  similarly  for  the  light 
collection. 20  We  suggest  that  our  normalization  could  proceed  independently  for  several 
subgroups  of  image  intensities,  all  members  of  a  subgroup  sharing  similar  lightness  values. 
Such  an  extension  of  our  normalization  scheme  would  tolerate  some  "bending"  of  the  grey 
locus,  as  occurs  when  a  record  is  doubled. 

It's  worth  asking  whether  any  natural  situations  cry  out  for  normalization  that  varies 
with  intensity,  such  as  we  propose.  That  is,  where  might  we  expect  the  spectral  character 
of  illumination  to  vary  with  intensity?  Aquatic  environments  come  to  mind.  At  a  given  depth, 
light  coming  from  bdlow  has  lower  intensity  and  lower  peak  wavelength  than  does  light 
coming  from  above  (Levine  &  MacNichol,  1982;  McFarland  &  Munz,  1975a).  Two  distinct 
Landspaces  could  be  set  up,  one  for  the  lower  visual  hemifield,  another  for  the  upper. 
In  terrestrial  habitats,  it  might  be  desirable  to  normalize  shadowed  (low  intensity)  regions 
separately  from  fully  lit  (high  intensity)  regions  in  the  same  image.  This  would  be  the  case 
if  the  diffuse  light  were  bluer  than  the  direct  light  due  to  scatter.11 


7.3  Neurophysiological  Operators 

How  might  operators  be  constructed  that  would  detect  crosspoints  and  the  opposite 
slope  sign  condition?  Consider  ciosspoints  first.  Suppose  we  would  like  to  detect 
crosspoints  of  the  sort  shown  in  Fig.  8a.  Two  spectral  samples,  M  and  L,  are  taken  on 
both  sides  .V  and  Y  of  an  edge.  Figure  8b  shows  schematically  such  a  detector.  Intensity 
values  of  the  I.  samples  are  compared  across  the  edge.  We  define  (£!)  as  the  difference 
between  the  I.  value  of  side  Y  from  the  /,  value  of  side  X,  if  that  value  is  positive,  and  zero 
otherwise.  Similarly,  the  M  samples  are  compared  across  the  edge.  Here  the  M  value  of 
side  .V  is  subtracted  from  that  of  side  Y  (again  with  a  zero  minimum).  The  results  of  the 


*°Of  course,  this  assumption  can  be  tested.  If  dark  materials,  say,  tended  to  be  reddish,  our 
assumption  would  be  incorrect  and  require  modification. 

•'Note  that  in  this  case,  the  grey  world  assumption  of  section  3.4  is  inappropriate. 
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Figure  S  Detecting  crosspoints  and  tho  opposite  slope  sign  condition,  a)  The  spectral  crosspoint 
to  lx-  detected,  b)  A  schematic  crosspoint  detector  Note  a  logical  spectral  operation  follows  an 
arithmetic  spatial  operation,  c)  Useful  intermediate  urn's  for  crosspoint  detection  arc  sketched.  Split 
bar  shaped  configuration  indicates  a  spatial  comparison.  Comparisons  are  made  within  spectral 
channels.  Outputs  of  those  intermediates  can  the.',  be  combined  logically,  d)  The  opposite  slope  sign 
condition  to  he  detected,  o)  A  schematic  dotoctor  for  the  opposite  slope  sign.  Note  a  logical  spatial 
operation  follows  an  arithmetic  spectral  operation,  f)  Useful  intermediate  units  for  opposite  slope  sign 
detection.  Spectral  comparisons  are  made  in  a  single  spatial  region.  Outputs  of  therje  intermediates 
can  then  bo  combined  logically. 
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two  arithmetical  operations  (J\ )  and  (£{,)  are  combined  logically  with  an  AND.22 

Next  consider  the  detection  of  opposite  slope  signs  as  shown  in  Fig.  8d.  On  each 
side  of  the  edge,  a  spectral  comparison  must  be  made.  The  arithmetic  operation  (/,+  M~) 

denotes  the  value  of  the  M  sample  subtracted  from  that  of  the  /,  sample  on  one  side  of  the 
edge.  This  operation  has  a  minimum  value  of  zero.  The  spectral  comparison  (/,'  M~)  is 

then  ANDed  with  the  similarly  defined  comparison  (//  A/4)  on  the  opposite  of  the  edge 

to  create  the  opposite  slope  sign  detector  shown  in  Fig.  8e. 

It's  worth  noting  that  for  the  crosspoint  detector,  two  spatial,  arithmetic  operations  are 
logically  combined  across  wavelength  samples.  For  the  opposite  slope  sign  detector,  two 
spectral,  arithmetic  operations  are  combined  logically  across  space  (across  an  edge) 

Both  detectors  (Figs.  8b  and  8e)  superficially  resemble  double  opponent  units  described 
in  many  species  (Daw,  1972;  Michael,  1978a;  Livingstone  &  Hubei,  1984).  That  is,  both 
detectors  have  two  spatial  fields— one  that  receives  excitatory  connections  from  one  spectral 
sample,  and  inhibitory  inputs  from  another  spectra!  sample,  and  a  second  spatial  field 
receiving  the  opposite  spectral  inputs.  Neurophysiological  tests  have  not  yet  been  sufficiently 
detailed  to  distinguish  whether  double  opponent  units  are  computing  spectral  crosspoints, 
or  opposite  slope  signs,  or  neither. 

Useful  intermediate  units  for  crosspoint  detectors  are  shown  in  Fig.  8c.  The  line  through 
the  bar-shaped  configuration  indicates  a  comparison  across  space.  Comparisons  are 
within  spectral  channels;  outputs  of  these  intermediates  can  be  comtined  logica'ly  to  build 
crosspoint  detectors.  This  sort  of  intermediate  unit — spatial  comparisons  in  isolated  spectral 
channels — has  only  been  reported  once  (Michael,  1978b),  apparently  without  replication. 

Figure  8f  depicts  useful  intermediate  units  for  opposite  slope  sign  detection.  Spectral 
comparisons  are  made  within  a  single  spatial  region.  Units  of  this  type— spectrally  opponent 
but  spatially  undifferentiated— have  been  described  by  physiologists  recording  from  monkey 
lateral  geniculate  nucleus  (Weisel  &  Hubei,  1966;  Kruger,  1977;  Michael,  1978b).  This 
evidence  suggests  that  the  double-  -opponent  units  described  in  primate  VI  might  be 
performing  an  opposite  slope  sign  computation.  Detailed  color  neurophysiology  is  needed 
to  test  this  notion. 


^“Strictly  speaking,  the  results  of  the  two  arithmetical  operations  must  be  converted  to  0  or  1 
before  being  logically  combined.  Alternatively,  the  results  of  the  arithmetical  operations— the  modified 
subtree  :ons — could  be  multiplied  together.  A  non-zero  product  implies  a  crosspoint. 
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8.0  Summary 

Our  theory  of  co'^r  vision  presents  two  types  of  operators— the  spectral  crosspoint  for 
normalization  and  the  opposite  slope  sign— which  suffice  in  most  cases  to  normalize  for  the 
illuminant  and  to  categorize  the  albedoes  in  the  scene.  Our  scheme  should  differentiate 
oetween  the  common  natural  pigments  (chlorophylls,  xanthephylls  and  flavanoids),  for 
e'  ample,  but  not  between  the  density  variations  of  any  one  of  these  pigments.  The  theory 
does  not  address  this  latter  problem— namely  how  we  appreciate  tne  fine  changes  in  the 
grain  of  a  piece  of  teakwood.  A  quantitative  color  vision  system,  of  greater  complexity  than 
the  qualitative  computations  described  here,  will  be  needed  for  such  fine  discriminations. 
Categorical  color  vision  is  simply  an  inexpensive  method  for  making  rapid  and  reliable 
coarse  judgments  about  materials 
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Appendix  1:  Lawful  Processes 

This  appendix  shows  that  image  edges  that  arise  from  1)  change  in  surface  orientation, 
2)  pigment  density  variations,  3)  shadows  and  4)  highlights  all  preserve  the  ordinal  relations 
of  image  intensities  across  that  edge,  and  hence  cannot  cause  the  opposite  slope  sign 
condition. 

1.0  Surface  Orientation  Change 

Let  .Y  and  Y  be  regions  on  either  side  of  an  edge  due  solely  to  a  surface  orientation 
discontinuity.  Then  the  image  intensities  (as  functions  of  wavelength)  /.v(\)  and  /y(X), 
measured  in  ,Y  and  respectively,  are  related  multiplicatively.  That  is,  /.v(X)  :  <*/>(X)  for 
some  constant  <«  (Rubin  &  Richards,  1982;  Horn  &  Sjoberg,  1979).  Two  functions  differing 
only  by  a  multiplicative  constant  have  identical  ordinality. 


2.0  Pigment  Density  Variation 

Suppose  X  and  V'  are  two  regions  on  a  planar  piece  of  a  single  material  that  differ 
only  in  pigment  density.  Then  if  the  albedo  (as  a  function  of  wavelength)  of  region  .Y  is  ^(X), 
the  albedo  of  Y  can  be  approximated'3  by  />6(X),  where  6  is  a  constant  related  to  pigment 
density  (Rubin  &  Richards,  1982;  Wyszecki  &  Stiles,  1967). 

The  light  measured  from  regions  .Y  and  Y  is  the  product  of  the  albedoes  of  ,Y  and  Y 
with  the  radiant  intensity  of  the  illuminant.  Since  X  and  >'  are  assumed  coplanar  (recall  that 
pigment  density  change  is  stipulated  as  the  sole  cause  of  the  edge),  and  the  illumination  is 
the  same  for  both,  then  any  difference  between  measured  intensities  from  the  two  regions 
will  be  due  to  a  difference  in  albedo  functions.  But  the  albedo  functions  are  related  by 
an  exponential  constant,  and  two  functions  so  related  have  identical  ordinality.  Therefore, 
image  intensities  across  a  pigment  density  change  will  have  identical  ordinality.  [Examples 
of  this  relation  for  natural  pigments  can  be  seen  in  Krinov  (1971),  Francis  &  Clydesdale 
(1975)  or  Snodderly  (1979).] 


3.0  Shadow 

Consider  an  edge  separating  a  lit  region  from  a  shaded  one.  Both  lit  and  shaded 
regions  reflect  diffuse  illumination  toward  the  viewer.  The  lit  region,  in  addition,  reflects  a 

::,This  exponential  relation  presumes  that  the  embedding  material  is  spectrally  neutral.  If  the 
embedding  layer  reflects  different  wavelengths  unequally,  then  change  in  pigment  density  has  a  more 
complex  description,  In  particular,  pigment  oensity  changes  can  mimic  material  changes. 
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direct  source  It  /„>{>•)  and  /.,  ,  ,iM  are  image  intensities  (as  functions  of  wavelength  X)  from 
lit  and  shaded  regions  respectively,  then 

/.<•  ,  ll  X  )  /  ix  MM  .  j. 

L'\ M  iu,!!.4.A\)-i:l„.,(\)r(  X) 

where  /  •>> < ...  ( x  >  and  I.  ,,.  . ( x  j  are  the  diffuse  and  direct  components  of  illumination,  and 
,'(X)  characterizes  the  albedo  of  the  material 

Dy  inspection  of  equations  (1).  it  is  clear  that  ordmality  can  be  violated  in  the  case 
of  shadow  That  is.  a  false  target  is  possible  The  visual  world  fortunately,  offers  certain 
regularities  There  is  usually  some  close  relation  between  diffuse  and  direct  illumination 
(Goral  vt  ,n  .  I  OB-1)  This  is  not  surprising  since  diffuse  light  results  from  diverse,  random 
reflections  of  the  direct  light  from  a  variety  of  materials  in  the  scene  An  assumption  will 
tie  made  that  this  is  usually  the  case  a  visual  system  can  presume  that  diffuse  light  fv  3 
the  same  spectral  character  as  the  direct  light  That  is.  ,(X)  k  I  , for  sor  >e 

constant  k.  This  wo  call  the  grey  world"  assumption  (see  Section  3  1).  because  t  is 
implied  by  the  statement  that  all  the  albedoes  of  a  scene  will  average  to  grey.  Anecd  tal 
data  support  the  grey  world  assumption  Mailman  (1970)  measured  spectral  irradis  ice 
functions  in  a  pine  woods  in  a  sunny  area  and  in  nearby  shade  The  functions  are  strike  gly 
similar  in  shape,  and  are  shown  in  Fig.  9. 

Invoking  the  grey  world  assumption,  equations  (1)  become: 

/.*..<(  M  . (MtfA)  (2) 

X)  (I  •  *)A*,,„(AMX) 

Note  th.it  the  lit  and  shaded  regions  now  give  rise  to  multiplicatively  related  image 
intensity  functions  Ordmality  will  therefore  be  preserved. 


4.0  Highlights 

The  analysis  of  highlights  is  slightly  more  complex  The  following  equations  (Rubin  A 
Richards  190?.  equations  M  i)  express  the  image  intensities  to  be  found  in  a  highlight  and 
neighboring  matte  region: 

/,,-i.(M  (/'|,//,.,,(X)  •  ,„,,(X))p(X) 

ltujhi,3ht(^)  t  *  •'./,».  rf(X)  -(!  (X)  •  *,.-,  ( X )  /.( X ) 

where  (X)  and  ^.(X)  are  the  images  intensities  (as  functions  of  wavelength)  in 
matte  and  highlighted  regions,  and  *  •  10  |)  is  a  constant  that  indicates  to  what  extent  the 
su.fuce  is  mirrorlike  (<*  I  describes  a  perfect  mirror)  (See  Richards.  Rubin  A  Hoffman, 
198.?,  for  a  more  extended  treatment.) 
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Shadow  and  Wavelength 


Figure  9  Measurements  of  the  spectral  irradionce  functions  of  direct  sunlight  and  nearby  shade  in  a 
Florida  pine  woods,  adapted  from  Mailman  (1979),  Fig.  74a.  On  the  ordinate  is  the  logarithm  of  photon 
#1  flux.  The  abscissa  shows  wavelength. 


The  equations  express  the  fact  that  both  highlighted  and  matte  regions  reflect  both  direct 
and  diffuse  light.  In  addition,  the  highlight,  acting  as  a  partial  mirror,  reflects  the  direct  light. 


Applying  the  grey  world  assumption,  equations  (3)  become: 


fm.l<rr(X)  --  (1  f  <,')/'-’difrrf(X)/»(X) 

f htjhltg  hf(X)  X /l,dvrrrf(X)  *'  (I  ^  )(  I  "T" 


(4) 


which  reduces  to 

f*nall/(M  (I  F 

-  /vrf.,.re(X)i«  Ml  “  W  ♦  *V(X)1 

By  inspecting  equations  (5),  it  can  be  seen  that  highlights  can  produce  a  spurious 
violation  in  ordinality.  Assume  now  that  the  image  has  been  normalized  with  respect  to  the 
color  of  the  illuminant.  Normalization  is  any  scheme  that  allows  recovery  of  the  spectral 
character  of  the  illuminant.  (Such  a  computation  is  presented  in  section  4.)  Normalization 
is  equivalent  to  a  transfcrmat.on  of  the  image  intensities  to  what  they  would  have  been  had 
the  illuminant  been  white;  it  allows  us  to  set  ICj,rrrt(\)  --  ft,  where  ft  is  some  constant. 

Both  equations  (5)  can  now  be  rewritten  substituting  ft  for  ((X),  yielding 
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/»,, •  (I  *)(« 

With  the  two  assumptions  of  grey  world  and  spectral  normalization,  highlights  will  not 
produce  -'oiations  in  ordinary  This  can  he  seen  in  equations  (6),  where  the  image  intensity 
function  of  the  highlighted  region  is  simply  related  to  the  image  intensify  function  of  the 
neighboring  matte  region  The  intensity  in  the  matte  region  is  multiplied  by  a  constant 
(l  ,■>).  and  then  a  constant  function  (/(X)  a.<)  is  added  These  two  operations  preserve 

ordmality;  hence  no  opposite  slopes  will  arise  given  our  assumptions. 
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Appendix  2: 

Algorithm  for  Spectral  Normalization 
and  Material  Categorization 


Given  a  full  color  image  of  a  scene  lit  by  an  unknown  illuminunt.  and  a  way  of  finding 
edges  and  regions,  regions  can  be  assigned  to  one  of  a  small  number  of  material  categories. 
Regions  in  different  categories  are  made  of  different  materials  An  algorithm  for  categorizing 
materials  is  sketched  below.  The  first  step  is  10  correct  for  colored  illumination;  the  second 
is  to  categorize. 


In  the  Beginning 

The  original  full-color  image  can  be  viewed  thropgh  three  spectral  filters,  yielding  three 
distinct  maps  of  image  intensity,  say  It,  G,  and  D.  See  Fig-  10a.  These  three  maps  of  image 
intensity  we  call  "spectral  images."  The  number  of  filters,  or  their  spectral  characteristics, 
should  not  be  important.  All  that  matters  is  that  the  filters  yield  independent  measurements. 

I 

Spectral  Normalization 

I 

First,  apply  an  edge  operator  to  the  image.  The  particular  edge  operator  should  not  be 
crucial.  Assume  the  edge  operator  produces  a  closed  set  of  edges.2'  Next,  edge  segments 
must  be  made  explicit.  See  Fig.  10b.  This  involves  linderstanding  vertices.  For  example,  a 
T-vertex  terminates  the  edge  that  is  the  leg,  but  not  the  edge  that’s  the  crossbar.  Identifying 
edge  segments  is  important  because  we  will  iterate  through  a  list  of  them. 

For  each  edge  segment,  two  narrow  strips  must  be  defined,  one  on  each  side.  Call 
the  strips  X  and  Y.  (Understanding  vertices  is  important  because  the  strips  must  be  free  of 
edges.)  See  Fig.  10c. 

Average  the  intensity  values  of  each  of  the  spectral  images  It,  <7,  and  D  in  both  the  .V 
and  Y  strips.  The  output  of  this  step  is  six  values  Rx,  lly,  Gx,  Gy,  nXl  and  Dr. 

For  each  edge  segment,  check  for  two  types  of  crosspoint,  ItG,  and  DGP  (The 
conditions  are  {llx  ~  lty){Gx  -  Gy)  <  0  and  ( Gx  -  Gy)(Dx  -  Dy)  <  0,  respectively.)  Note 
the  possibility  of  a  third  crosspoint  involving  the  It  and  /?  samples. 

Suppose  an  image  has  n  crosspoint  edge  segments.  For  each  crosspoint,  record 

•'If  algorithm  for  edge  detection  does  not  produce  closed  edges,  then  regions  must  somehow  be 
identified  using  edge  fragments. 

'•'"The  It  and  C  samples  can  yield  crosspoints,  and  independently,  so  can  the  II  and  (1  samples.  The 
(l  sample  could  just  as  easily  bo  taken  as  the  photopic  luminosity  function. 
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Figure  10  a)  The  full-color  image  is  run  through  three  spectral  filters  It.  (1.  and  II.  b)  Fdgo  segments 
nave  been  found  and  made  explicit.  This  image  shows  five  edge  segments.  Vertices  have  been  found, 
and  are  hero  marked  with  largo  black  dots,  cj  On  either  side  of  one  of  the  edges,  narrow  strips  V  and 
)'  arc  detmed.  No  edge  segments  should  tic  in  the  strips.  Intensity  averages  will  be  taken  in  the  three 
spectral  images  in  botti  of  the  snips,  yielding  six  measurements.  This  is  done  for  each  edge  segment 
in  (lie  image,  d)  Measurements  taken  Irom  strips  about  each  edge  map  to  points  in  a  S()eclral  space 
defined  by  axes  as  labeled.  Normalization  consists  of  multiplying  It  and  II  v, dues  by  factors  such  that 
equal  numbers  of  points  will  bo  found  in  cacti  quadrant.  _ _ _  _ _ _  _ 
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spectral  information  about  the  two  abutting  strips.  In  particular,  store  two  color  contrast 
values  per  region. 


it,  a,  n,  a. 

It,  G,'  lii  +  Gi' 


(8) 


where  i  is  an  index  ranging  over  the  2n  edge  strips  defined  around  n  crosspoints.  This 
particular  form  of  ratio  is  useful  because  its  value  must  lie  in  the  closed  interval  [-1,1].  The 
spectral  information  recorded  can  be  considered  as  2 «  points  in  a  two-dimensional  spectral 
space  (with  axes  of  and  shown  in  Fig.  lOd.  (See  also  Fig.  4a.) 

Let  U  be  the  number  of  points  in  the  upper  half  plane  of  the  spectral  space  (Fig.  lOd), 
and  L  be  the  number  of  points  in  the  left  half-plane.  Under  a  white  illuminant,  we’d  expect 
a  random  assortment  of  materials  to  yield  U  L  n;  that  is,  points  should  be  roughly 
equally  distributed  among  the  quadrants  of  the  spectral  space. 

If  the  2m  points  are  not  divided  equally  among  the  quadrants  of  the  spectral  space,  we 
must  seek  normalization  constants  a  and  0  that  satisfy  the  following  criterion: 


MEDIAN 


alt,  —  G, 
all,  4  G\ 


i  =  l 


. Jn 


=  MEDIAN 


(Mi  -  Gi 
PUi  +  C, 


For  a  large  enough  number  of  image  regions,  we  can  take 


(9) 


l-U 


RG 


0  = 


1  ~  &RG 


1  +  &RG  1  +  C'flG 

where  Urg  and  Cur,  are  means  of  the  sets  of  measurements  (8): 


(10) 


7,  I  V*  fij-Cj  7,  1  ^  Di  -  Gi 

RG  ~  2 n  Ri  +  Gi  VaG  ~  2 n  £*  +  G{ 


(11) 


The  values  of  a  and  0  in  (10)  will  provide  a  correct  normalization  (i.e.,  normalization 
criterion  (9)  will  hold)  given  some  simple  statistical  conditions.28 

The  correctness  of  the  normalization  constants  a  and  0  can  easily  be  checked  by 
verifying  that  criterion  (9)  holds.  If  not,  the  values  of  a  and  0  can  be  adjusted  incrementally 
in  an  iterative  procedure.  The  entire  normalization  algorithm  is  shown  as  a  flowchart  in  Fig. 
11. 

Once  correct  values  of  the  normalization  constants  are  returned  by  the  algorithm,  the 
three  spectral  images  It,  G  and  D  can  be  transformed  into  a  set  of  normalized  spectral 

'•'’There  must  be  at  least  12  independent  crosspoint  edges,  and  the  mean  and  median  of  the  set 
of  measurements  {  }  must  approach  the  same  value  as  »'  >-♦  oo,  and  similarly  for  the  set  of 

measurements  {  }.  (See  Siegel,  1956.) 
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NORMALIZATION  ALGORITHM 


Begin  with  Points 
in  Spectral  Space 
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Set  a  = 


2n  p  _g 
_ i*t _ _ 

1  +1/2n  J 

i*1 
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2n  B,-6, 
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_ 1*1 _ 

2n  e,-6i 
1*1/2"  1  FTF 
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I  Compute  d  and  U  1 

__ jizzz. 
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Increment  a 


4r>n 


Oecremen?  a  l*-i 


Compute 

aR-G 

aR*G 

and  recompute 
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Compare 

ft; 

Stop: 

Stop: 

Compare 

U  and  n 

Return  a 

Return  /? 

U^andn 

Increment  )3 


pwj  Decrement/? 


«zf  <i» 


Compute 

$s-a 

/9B+G 

and  recompute 

X 


f  t'jure  1 1  Normalization  Flowchart,  Begin  with  points  scattered  in  spectral  space,  and  end  with 
a  pan  of  multiplicative  normalization  coefficients,  <»  to  balance  the  II  image  with  respect  to  C,  and  /? 
to  balance  the  II  imago  with  respect  to  O’. _ _ _ 
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(A)  (B) 


R 

G 

B 

Ixa 

T 

R* 

G* 

B* 

Figure  1 2  a)  The  three  spectral  images  It,  G,  and  II  are  normalized  using  the  multiplicative  constants 
produced  by  the  procedure  shown  in  Fig.  11.  The  normalized  spectral  intensity  maps  are  It',  G",  and 
ll‘.  b)  The  regions  of  the  image  sketched  in  Fig.  10b  labeled  with  material  categories.  Each  region  is 
assigned  one  of  four  possible  ordinal  doublets. 


wages.  All  values  in  the  It  image  are  multiplied  by  »,  yielding  It".  (The  asterisk  superscript 
denotes  normalized  intensity;  see  section  7.2.2.)  Similarly,  IV  pH.  Spectral  image  G  is 
unchanged:  G  —  G‘.  See  Fig.  12a. 


Spectral  Categories 

Suppose  that  when  closed  edge  segments  were  found  that  image  regions  were  made 
explicit.  For  each  region  i,  measure  the  average  values  of  the  normalized  spectral  images, 
yielding  the  triplet  (/?*,  <7*,  /#*).  A  triplet  of  numbers  yields  one  obvious  pair  of  ordinal 
relations: 


(/?*, Gill')  *-►  (sign lie, <*ignUa)i 


where  *ignn<;  is  "+”  if  G *  >  III  and  otherwise. 

Each  region  can  therefore  be  assigned  to  one  of  four  material  categories:  ( t- ,  +),  (-,  4-), 
(-,+),  (•*-,-).  This  is  shown  in  Fig.  12b.  Two  regions  that  are  in  different  categories  a(e 
composed  of  distinct  materials. 

Note  that  a  third  ordinal  relation  is  sometimes  independent,  the  It'  -  li'  comparison^. 
If  this  relation  is  included,  six  spectral  categories  obtain. 


Finally,  note  that  while  the  algorithm  described  here  is  categorical,  continuous  infor¬ 
mation  has  not  been  lost;  it  is  stili  available  for  more  refined  purposes.  For  each  region  i, 
the  continuous-valued  coordinates 
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Ri  -Ci  H,  -  <7, 
+  a]'  ft +Gi 


(13) 


should  be  useful. 
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